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ABSTRACT 



In 1996, the Department of Indian Affairs and Northern 
Development (DIAND) Youth Strategy introduced summer career placement 
programs, science and technology camps, and cooperative education programs 
for First Nations and Inuit youth in Canada. This evaluation focuses on the 
initial year of implementation, and is based on 100 telephone interviews and 
community visits with First Nations stakeholders, and interviews and visits 
to DIAND headquarters and regional offices. Programs were administered by 
DIAND, First Nations and Inuit organizations, or a combination of those two 
approaches. First Nations and Inuit administration was the most effective due 
to expertise in youth programming, use of existing infrastructure, and less 
time and money spent on administration. Students were exposed to a wide range 
of sectors, where they acquired basic job skills and increased their 
self-esteem, knowledge of job markets, interest in studies, and desire to 
stay in school . The evaluation presents lessons learned and improvements 
suggested by First Nation and Inuit participants. Overall, the initial 
implementation was considered very successful, with 3,800 students 
participating in summer programs, 2,000 students attending science camps, and 
900 students participating in cooperative education. Recommendations focus on 
encouraging and providing financial support for program management by First 
Nations and Inuit regional organizations, improving reporting, and measuring 
long-term impacts of the strategy. Appendices present a summary of findings, 
regional reviews and case studies, a statistical overview, and an evaluation 
f ramework . ( TD ) 
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Executive Summary 



In 1996-1997, DIAND became a partner in the federal government’s national Youth Employment 
Strategy and received $10 million. The strategy has been renewed for 1997-1998 and 1998-1999 
with two additional programs and funding of $25 million per year. This interim evaluation 
focuses on the initial year of implementation examining the effectiveness of the strategy and its 
short-term impacts. 



Purpose of the Evaluation and Approach 

DIAND has undertaken to submit to Treasury Board an interim evaluation of the strategy. The 
purpose of this evaluation is to examine the delivery of the strategy’s programs, the effectiveness 
of the initial implementation, and short-term impacts. The evaluation is formative, with a central 
focus on lessons learned and potential improvements. A supplementary product of the report is 
an evaluation framework which sets out an accountability and performance framework to help 
DIAND, First Nations and Inuit assess long-term impacts. 

In the spring of 1997, we conducted just under 100 telephone interviews with First Nations 
stakeholders and visited three communities (part of which included 3 focus groups with youth). 
We also visited DIAND headquarters and three regional offices (Manitoba. Saskatchewan and 
Quebec) to conduct interviews and to review file data The team also interviewed DIAND 
regional youth coordinators. 



Findings 

Status of Program Implementation 

In spite of the short time between the strategy’s announcement and the end of the school year. 
First Nations successfully implemented the Summer Student Career Placement and Science and 
Technology Camps programs. First Nations schools began to develop their Cooperative 
Education Program in the fall of 1996, once school started. Just under half the First Nations 
schools we interviewed indicated that their programs began in September 1996 or January 1997. 
A third of the schools reported expanding an existing program while the remainder were in the 
planning stages. 
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Overall Effectiveness of the Strategy’s Implementation 

Strategy implementation varied across the regions, but 3 approaches emerged: 

1 - DIAND administration of programs; 

2 - First Nations and Inuit organizations administration of programs; and 

3 - DIAND and First Nations and Inuit administration of programs. 

Regions such as Manitoba and the Yukon are directly involved in reviewing proposals, allocating 
funding, and monitoring program reporting. In Saskatchewan and Quebec, First Nations 
organizations manage the strategy on behalf of communities. The remaining regions pursued a 
combination of the two approaches. First Nations and Inuit administration of the strategy was 
found to be the most effective for reasons such as expertise in youth programming and use of 
existing infrastructure to distribute and share information. A review of program reporting data 
found that regional First Nations organizations also spent less time on managing the strategy and 
distribute a higher percentage of funds directly to students. They also have a higher rate of 
reporting. 

First Nations and Inuit at the local level are continually pursuing partnerships with government 
departments and the private sector. Overall, First Nations and Inuit, especially northern and 
isolated communities, rely on existing on-reserve organizations to provide youth with 
opportunities. First Nations and Inuit also contribute a substantial amount of additional funds to 
the strategy. 

While respondents believe that the strategy has met its objectives, under-reporting does not allow 
the department to accurately reflect these successes. For example, in regions such as British 
Columbia and the Yukon, First Nation submissions were less than half complete. In some cases, 
reporting is also inconsistent and incomplete. 

Reported Short-Term Impacts 

Most respondents believe the strategy has had positive impacts. To date. 

• 3,800 students participated in summer student programs; 

• 2,000 students took part in science camps; and 

• 900 students participated in cooperative education. 

Students were placed in a range of sectors, such as health, education, construction, retail and 
economic development. Students acquired basic job skills (computers, administration, sales, 
communication) that they can use in all fields. In focus groups, youth also stated that they 
acquired preparation skills (interviewing, writing cover letters and developing resumes) that will 
prepare them to compete for future jobs. 

• Elementary and high school students were exposed to science and technology camps 
ranging from experiences in traditional medicines to university computer labs 
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. Students reported that the strategy’s programs had a positive personal influence. They 
reported an increase in self-esteem, improved knowledge of the job market, an enhanced 
interest in their studies, and a desire to stay in school 

Identifying Indicators to Measure Long-Ter m Impacts 

Since the evaluation focussed on the initial year of implementation, it is too early to measure long- 
term impacts. However, the evaluation did assess what information is currently being collected 
and how it can be used to measure long-term results and ongoing performance monitoring. At 
present, program evaluation forms are used to collect data on placements and participants, while 
First Nations and Inuit informally collect data on program activities. 

A supplementary product of the study is an evaluation framework which sets out an accountability 
and performance regime to help DIAND and First Nations and Inuit assess long-term impacts. 

The framework identifies many indicators for decision-making and evaluation purposes. 

Lesson Learned and Suggested Improvements 

Based upon the experiences of First Nation, Inuit and departmental officials, the evaluation 
identifies a number of lessons learned in terms of implementation, communication, effectiveness, 
partnerships and community support, and reporting. Suggested improvements identified by First 
Nation and Inuit participants for their own program implementation include selective matching of 
students and employers, obtaining community support, integrating the strategy’s programs into 
the communities’ economic development plans, including Elders in the process, developing 
orientation sessions for local coordinators, and having dedicated coordinators for cooperative 
education programs. 



Recommendations 

Overall the initial implementation of the strategy was viewed to be very successful. First Nation 
and Inuit implementing organizations noted concerns over the level of resources being devoted to 
the administration of the strategy’s programs. The recommendations of the evaluation are: 

• it is recommended that increased activities and resources be devoted to capacity building 
among First Nations and Inuit and related organizations to support them in implementing 
the strategy. 

• it is recommended that all DIAND regional offices continue their efforts to encourage 
First Nations and Inuit regional organizations to adopt the management of the strategy. 

To assist this, the department may need to consider providing a financial support to First 
Nation organizations to administer the strategy, as has been done in the Quebec Region. 
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• DIAND regional offices should ensure that annual First Nations and lnuit audit reports are 
tracking Youth Strategy expenditures. 

• it is recommended that the timeliness and level of submissions of program evaluation 
forms be improved. 

• the evaluation forms should be enhanced to include information on the type and number 
of employers participating in the strategy, the types of skills/positions undertaken, and the 
sources of additional funds contributed to the programs. 

• the department should work with First Nations and lnuit and other departments involved 
in Aboriginal youth programming to promote consistency in the reporting requirements 
across federal departments. 

• it is recommended that DIAND identify, in consultation with First Nations and lnuit, 
performance measurement indicators that will assist both partners to measure the long- 
term success of the strategy. 

• for the long-term evaluation of the strategy, it is recommended that a partnership approach 
be undertaken by the department and First Nations and lnuit. This process should include 
Aboriginal youth and use the evaluation framework developed in the course of this current 
interim evaluation as a starting point. 
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Section 1 - Introduction 



Purpose of the Report 

In March 1 996 the Government of Canada announced that it would spend S3 1 5M over three 
years to help create employment opportunities through the Youth Employment strategy’ Human 
Resources Development Canada is the lead federal department with partners such as Industry 
Canada Environment Canada, Canadian Heritage, and the department of Indian Affairs and 
Northern Development (DIAND). DIAND was a partner in the strategy for 1996-1997 and 
received S10M for programming. The strategy was renewed for an additional two years with 
funding totalling $25M per year. 

A review after one year of operation presents an opportune time to review lessons learned to date 
and to identify mid-term corrections for future programming. This report summarizes the findings 
of the interim evaluation of the DIAND Y outh Strategy. 



Purpose of the Evaluation 

DIAND has undertaken to submit to Treasury Board an interim evaluation of the strategy. The 
review examines the delivery of the programs, the effectiveness of the strategy’s initial 
implementation, and short-term impacts. The central focus of the evaluation is on lessons learned 
and potential improvements. A supplementary product of the report is an evaluation framework 
which sets out an accountability and performance regime to help DIAND and First Nations assess 
long-term impacts. 



Evaluation Scope and Issues 

The evaluation focuses on the initial strategy and is limited in scope to the three programs 
originally targeted to First Nations and Inuit communities (See Figure 1). The evaluation does 
not deal with the two new programs added in the spring of 1997. However, profiles for each 
program are discussed in the evaluation framework attached to this report. 

The terms of reference for the evaluation were approved by the Departmental Audit and 
Evaluation Committee on October 31, 1996 (See Annex A). The evaluation issues to be 
addressed are: 

• what is the status of program implementation? 

• how effective and efficient has the initial strategy implementation been? 

• what have been the short-term impacts of the strategy? 
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• What are the lessons learned and best practices' 7 

• What improvements can be made and what are the options for future programming' 7 

What is covered in the interim evaluation? 



In 1996-1997 DIAND Youth Strategy 
consists of 3 programs: 

• First Nations & Inuit Summer 
Student Career Placement Program 

• First Nations & Inuit Science & 
Technology Camp Program 

• First Nations Cooperative 
Education Program 

* 1 — 

Focus 

of the interim 
evaluation 



In 1997-1998 and 1998-1999, j 
2 new programs are added: 



• First Nations and Inuit Youth 
Business Program 

• First Nations and Inuit Work 
Experience Program 




Focus of 
evaluation 
framework 



FIGURE 1 



Methodology 

Between February and May 1997, we used several methodologies to collect data, including: 

• telephone interviews with 95 First Nations stakeholders. We obtained a 92% response 
rate for which a breakdown by category is illustrated in Figure 2; 

• regional visits and a case study of a First Nation community in Saskatchewan, Manitoba, 
and Quebec; 

• as part of the case studies of communities, 3 focus groups with 2 1 First Nations students 
in Manitoba and Saskatchewan 1 ; 

• in-person interviews with DIAND headquarters staff and telephone interviews with 
regional youth coordinators; and, 

• file reviews at DIAND headquarters and the regional offices visited. 



We attempted to conduct additional focus groups in other regions but First 
Nations communities felt it was too early to measure program results and were 
reluctant to allowing their youth to participate in focus group discussions. 
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First Nations and Inuit Stakeholder Interviews 

95 Interviews out of 103 Contacted 



Response Rates Breakdown of 95 

Responses by category 



I FNC ; 




Number of First Nations and Inuit Stakeholder Interviews 



i First Nations and Inuit coordinators (FNC) 31 

j Participating Schools (PS) 26 

Non-Participating Schools (NPS) 28 

j FN Implementing Organizations (FNIO) 10 



Organization of the Report 



This report is divided into several sections: 



Section 1 - Introduction 
Section 2 - Program Profile 

Section 3 - Status of the Initial Program Implementation 

Section 4 - Effectiveness of Strategy Implementation 

Section 5 - Measuring Impacts 

Section 6 - Lessons Learned/Best Practices 

Section 7 - Suggested Improvements 

Section 8 - Conclusions and Recommendations 



We have appended to this study the terms of reference for the evaluation and separately bound 
reports with the findings of our data collection. These appendices include findings from the 
interviews with First Nations and Inuit stakeholders, case studies, a statistical overview, and an 
evaluation framework. 
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Section 2 - Program Profile 



Background on the Strategy 

Youth have a difficult time making the transition from school to employment. Figure 3 shows 
that the youth unemployment rate in Canada has risen over the last several years. In May 1997, 
the unemployment rate was 1 7.2% for youth between the ages of 1 5 to 24 years, almost double 
that of Canadians over the age of 24 years 2 . 



National Unemployment Rate for Youth 
25 -r 




1981 1983 1985 1987 1989 1991 1993 1995 1997 

Although the most recent figures are not yet available, we know that the situation on-reserve is 
worse. Figure 4 shows the participation rate (percentage of total population in labour force) on- 
reserve is lower for registered Indians on and off-reserve between the ages of 15-24 years 3 . 

Figure 5 shows that the unemployment rate (percentage unemployed of total population) 
increases for all populations on-reserve. However, the rate is still highest for registered Indians; 
Census data from 1991 indicate that the unemployment rate for registered Indians on-reserve 
between the ages of 15 to 24 years was 43.7%. 



Statistics Canada CANSIM data. On-line service, June 1997. 

1991 Census Data, Aboriginal Peoples Survey, Table 1.3, Ages 15-24 years. 
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Participation Rate for Ages 15-24 years 



Unemployment Rate for Ages 15-24 years 
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FIGURE 4 FIGURE 5 

The Government identified the issue of youth unemployment in the February 1996 Throne Speech 
and appointed a Ministerial Task Force to review the problem The Task Force found that the 
youth most at risk include those with minimal education and facing multiple barriers such as poor 
literacy, disabilities, single parenthood and residing in remote locations. Many Aboriginal and 
Inuit youth face all of these problems and others, such as high suicide rates. First Nations youth 
unemployment is among the highest in Canada. In addition, the First Nations and Inuit youth 
population, which represents a third of the total Aboriginal population 4 , is increasing, thereby 
further pressing the urgency for Aboriginal and Inuit youth programming. 

Consulting with Canadian youth and employers, the Task Force found the major barrier to success 
was the “first job/experience conundrum ”. Another problem cited was inadequate information 
about the labour market and educational choices. The Government of Canada responded to these 
concerns by introducing the national Youth Employment Strategy which aims to provide youth 
with employment opportunities in partnership with the private sector and non-government 
agencies. 

Human Resources Development Canada (HRDC) is the lead department and its most popular 
youth programs. Youth Services Canada and Summer Career Placements, target urban Aboriginal 
groups and youth off-reserve. However, there were few programs specifically targeted to First 
Nations and Inuit youth. This prompted DIAND to become a partner in the strategy and introduce 
programs specifically targeted to these populations. 



Departmental Audit and Evaluation Branch, DIAND, Statistical Overview (Draft), 
June 1997. i tr 
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Profile of the Initial Strategy and the Recent Additions 

This section provides an overview of the original strategy and the new programs added during 
1997-1998. 

Programs and Objectives 

DIAND’s Youth Strategy is targeted to First Nations on-reserve and Inuit youth at the 
elementary, secondary, and post-secondary levels. The objectives of the strategy are to: 

• assist students in obtaining career related work experience; 

• promote science and technology as a career opportunity; and 

• establish or enhance existing cooperative education programs. 

To this end, DIAND introduced three programs to meet these objectives: 

• First Nations and Inuit Summer Student Career Placement Program 

• First Nations and Inuit Science and Technology Camps 5 

• First Nations Schools Cooperative Education Program 

Table 1 describes the programs, objectives, and guidelines. In the spring of 1997, the government 
renewed the strategy and introduced two additional programs which are described in Table 2: 

• First Nations and Inuit Youth Business Program; and 

• First Nations and Inuit Work Experience Program. 



This program was originally called the First Nations and Inuit Science and 
Technology Summer Camps Program. 
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Profile of Inilinl DIAND Youth Strategy Programs 
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Roles. Responsibilities and Stakeholders 



The theme of the strategy is to provide opportunities in partnership. Table 3 outlines the main 
roles of the strategy’s stakeholder groups. 



Table 3 



DIAND Youth Strategy Stakeholders and Main Roles 


DIAND 


First Nations 
Implementing 
Organizations* 


First Nations 


Employers 


Headquarters 

• policy and design 

• communicate information 

• develop criteria and 
guidelines 

Regions 

• allocate funding 

• review proposals 

• monitor and reporting 


• regional 
communication 

• develop funding 
proposals 

• coordinate 
activities 

• develop and design 
programs 

• submit repons 

• allocate funding 

• review proposals 

• monitor and 
reporting 


In addition to similar activities as 

FN Implementing Organizations: 

• develop partnerships with First 
Nations organizations/agencies 

• develop private sector on/off 
reserve partnerships 

• match students and employers 

• develop or seek out existing 
science camps 

• develop or enhance cooperative 
education program 


• submit 
proposals 

• provide 
employment 
opportunities 


First Nations and Inuit Implementing Organizations include education agencies, regional governments. Tribal 
Councils, Community Economic Development Organizations, etc. 

First Nations include schools, education authorities, economic development corporations, etc. 

(*) Note: There was a degree of difference in the extent to which these organizations undertook various 

implementation and administration activities. 



Program Funding 

The 1996-1997 strategy allocated a total of $10M as follows 7 : 

• $6,000,000 - Summer Student Career Placement Program; 

• $800,000 - Science and Technology Summer Camp Program; and 

• $3,200,000 - First Nations Schools Cooperative Education Program. 

Funding is based on the number of eligible students in each region for the summer employment 
and science camp programs. For Cooperative Education, funding is based on the number of 
students in schools on-reserve. 



DIAND headquarters data, February 1997. 2 1 
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A table with regional program funding allocations is found in annex. 

A total of $25M annually for 1997-1998 and 1998-1999 is allocated to the renewed strategy 8 . 
Table 4 provides details about the allocation for each program. 



Table 4 



Annual Program Allocations for the Renewed DIAND Youth Strategy 


$8,200,000 


First Nations and Inuit Summer Student Career Placement 
Program 


$1,800,000 


First Nations and Inuit Science and Technology Camps 


$6,000,000 


First Nations Schools Cooperative Education Program 


$1,500,000 


First Nations and Inuit Youth Business Program 


$6,500,000 


First Nations and Inuit Work Experience Program 


$1,000,000 


Housing Internship Program 9 



Reporting Requirements 

There are two types of reporting for the strategy: 

• Program Reporting - Information on outputs is submitted through the evaluation forms 
for each of the strategy’s programs. These reporting requirements include the number of 
participants, their ages, grades and gender. The Cooperative Education Program started 
later in the fall of 1996 and the department requested interim progress reports be 
submitted by the spring of 1997. 

• Financial Audit - At the end of each year, First Nations and Inuit submit an audit as part 
of the terms and conditions of their funding arrangement. The audit also contains 
information on how the strategy dollars have been spent. 



1 Source: D1AND web site (www.inac.gc.ca), April 1997. 

9 The Housing Internship Program is managed by the Canada Mortgage and 

Housing Corporation. It includes Inuit youth through four projects in NWT with 
Arviat, Pond Inlet, Arctic Bay and Baker Lake Housing Associations. 

oo 
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Section 3 - Status of Initial Program Implementation 



This section provides an overview of the process and results involved in the implementation of the 
DIAND Youth Strategy. 



Key Steps in Implementation 

Implementation of the strategy began in April 1996. Figure 6 identifies the key steps involved in 
implementing and delivering the strategy. 

. The Summer Student Career Placement and Science and Technology Camp programs 

were implemented in the summer of 1996. In some regions, camps were also held in the 
fall and winter. 

• The Cooperative Education Program was introduced later in the fall and winter. Our 
interviews with 25 participating First Nations schools indicated that 10 of the programs 
started in September or January 1997, 7 expanded an existing program, and the remainder 
were in the development and planning stages. 

Communication 

Regions sent letters and applications for proposals to First Nations councils and Inuit 
coordinators/organizations and Education Directors. Some regions distributed headquarters 
information packages to First Nations and Inuit organizations. In Quebec, Saskatchewan, and 
British Columbia, regional First Nations organizations developed information booklets, and held 
workshops or meetings to announce the programs. 



O o 
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Figure 6 



The Implementation and Delivery Process 




24 
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Guidelines 



Interviews with DIAND regional staff. First Nations and their organizations indicate that overall, 
most regions followed the national guidelines. However, many communities adapted the 
guidelines to meet their needs. For example, the Summer Student Career Placement Program 
guidelines state summer employment should be between 6 and 1 6 weeks, but many communities 
had 4-week terms to allow more students to gain experience. Many communities could not 
implement science camps in the strategy’s short time span and opted for fall camps. Headquarters 
later amended the guidelines to allow for year round camps. 

Program Fundine 

Interviews and file reviews indicate that most regions flowed funds for the programs to districts or 
First Nations and Inuit based on the national formula. Manitoba, Alberta and British Columbia 
funded the Summer Student Career Placement Program using a per capita formula to ensure that 
all communities could hire at least one student at the secondary and post-secondary levels. 

Some regions spent additional funds from their own budgets on the strategy. For example, the 
Quebec Region provided money to First Nations organizations to administer the strategy and 
Saskatchewan funded a post-secondary employment program that existed prior to the strategy. 






■ p \ 
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Regional Delivery Models 



Through interviews and case studies, we 
identified 3 administrative models for 
implementing the Strategy illustrated in 
Figures 7 to 9. 

Model 1: Region administers Strategy In 
the Yukon, Manitoba and Atlantic, the 
region reviews proposals, allocates funds, 
monitors evaluation forms, and reports on 
program outputs. 



Model 1 

Region Administers 
Strategy 

Region provides funding, 
reviews proposals, and evaluations. 




First Nations/lmiit Communities 

Manitoba, Yukon, and Atlantic 



Model 2: First Nations organizations 
administer the Strategy In Quebec and 
Saskatchewan, First Nations organizations 
with a regional mandate in education 
allocate funds, review proposals, provide 
advice, and monitor reporting. Since the 
renewal, British Columbia now also fits 
into this model. 

Model 3: A combination of regional & 
First Nations administration In British 
Columbia, Ontario, Alberta and the 
Northwest Territories, the region is 
responsible for one or more programs 
while others are administered by a First 
Nations or Inuit organization. 



FIGURE 7 



Model 2 

First Nations Organization 
Administers Strategy 



First Nations/lnuir “ImplementinL 
Organizations” administer Strategy^ 
review proposals, allocate funds, 

&. collect evaluation data. 



Quebec and Saskatchewan 




Region provides funding to 
“implementing organization*." 



First 

Nations/lnuit 

Communities 






FIGURE 8 



Model 3 

Region & First Nations 
Administration 



Region provides funding to 
First Nations and administers 
certain Strategy programs. 




c } 



First 

Nations/ Im 
Communiti 



British Columbia, Ontario, Alberta, and NWT 



BEST COPY AVAILABLE 





FIGURE 9 
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Table 5 provides a brief overview of the approaches used in each region. 

Table 5 



Overview of Regional Approaches to Administer the strategy 


Region 


Description of Approach 


Atlantic 


The region administers Uie strategy while education authorities and economic 
development corporations deliver programs. 


Quebec 


Five regional First Nations and Inuit organizations administer Uie strategy: the 
Conseil en Education des Premieres Nations. Institut Culturel et Educatif 
Montagnais. Cree Regional Authority, Administration Regionale Kativak. Bande 
Naskapi du Quebec. 


Ontario 


The DIAND region and districts. Tribal Councils and other First Nations 
organizations administer various strategy programs. 


Manitoba 


The region administers Uie strategy . 


Saskatchewan 


The strategy is administered by one First Nations organization. The 
Saskatchewan Federation of Indian Nations manages all three programs. 


Alberta 


One treaty organization administers one program but was in the process of 
returning the program back to the region. The other two programs are 
administered by the region. 


British Columbia 


The First Nations Education Steering Committee (FNESC) / First Nations Schools 
Association (FNSA) administered the Summer Student Career Placement and 
Science and Technology Camp programs. It is now responsible for the entire 
strategy. 


Yukon 


The region administers the strategy with field services officers who prepare 
proposals. There is no Cooperative Education Program 


Northwest 

Territories 


The region administers a Uiird of the budget while development corporations and 
Tribal Councils oversee two-thirds There is no Cooperative Education Program. 



Results/Outputs Achieved 

As of June 1997, it was estimated that a total of approximately 4,700 students participated in the 
strategy 10 . Table 6, provides more information on program outputs. 

• just under 3,800 students participated in the Summer Student Career Placement Program, the 
largest number coming from Ontario, Manitoba, and Alberta; 

• over 2,000 students took part in the science camps; and 



All regions had not yet reported; therefore, the information on program outputs is 
incomplete. 
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it is estimated that to date almost 900 students participated in the Cooperative Education 
Program. 






Table 6 



National Highlights of Re 


ported Program Outputs 


Program 


Number of 
Participants 


Age of Students 


Grade Level 


First Nations and Inuit 
Summer Student Career 
Placement 


• 3768 students 

• 66% 4-8 weeks 
placements 


• 61% ages 

15-19 years 


• 71% high school 
students 

• 20% have some post- 
secondary/ university 

• 1 0% elementary school 
students 


First Nations and Inuit 
Science and 
Technology Camps 


• 2032 students 

• 56% students from 
MB, AB and BC 


• 61% ages 
12-13 years 


• 36% less than grade 6 


First Nations Schools 
Cooperative Education 


• 879 students 

• 114 schools 


• mainly 

students ages 
15-19 years 


• mostly high school 
students 


Source: Departmental program reports, June 1997. 



Table 7 identifies how many schools are participating in the Cooperative Education Program. 



Table 7 



Number of Reported First Nations Schools Participating in the Cooperative Education 

Program 

(n=114) 


Atlantic 


Quebec 


Ontario 


Manitoba 


Saskatchewan 


Alberta 


British 

Columbia 


4 


38 


19 


11 


10 


23 


9 


Source: Departmental program reports, June 1997. 
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Section 4 - Effectiveness of Strategy Implementation 



The strategy was implemented over the past year; therefore, the evaluation discusses short- term 
program effectiveness. The following questions formed the basis of inquiry for the study and each 
will be discussed as it relates to the effectiveness of implementation, and to a lesser extent how 
efficient the strategy has been at achieving its objectives. 

• Were the objectives of the strategy achieved? 

• Did communication lead to awareness of the strategy’s programs? 

• Which approach led to the strategy being administered in a simplified, quick, and useful 
manner? Do regional differences have an effect on the implementation of the strategy? 

• Was reporting timely and consistent? 

• What partnerships have been developed? 

• What are the strategy’s strengths and weaknesses? 

Baseline data examining the effect of the strategy over several years is needed to make long- term 
conclusions. Section 5 and the evaluation framework accompanying this report (Appendix 9) 
further discuss which data to collect and how to use it. However, at this time, we can identify 
factors that promote cost-effectiveness and efficiency, such as delivery models, communication 
tools, streamlined administration, and coordination at the local level. 



Strategy’s Objectives 

Although the strategy has been in place for a year and actual impacts may not be observed for 
years to come, the majority of First Nations and Inuit respondents interviewed believe the 
strategy’s objectives were largely met. We interviewed 54 First Nations stakeholders 
participating in the strategy, including First Nations coordinators. First Nations organizations 
implementing the strategy, and schools that participated in the Cooperative Education Program. 
Table 8 indicates that the majority of these respondents believe the strategy has met the objectives 
of instilling self-esteem in students and providing work experience. Many respondents believe 
that it is not yet possible to assess longer term objectives such as increasing graduation rates or 
helping with the transition from school to work. 



O 
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Table 8 



What impacts do you anticipate the strategy will have? 

(n=54) 




Implementing 

Organizations 

n-10 


First Nations 
Coordinators 
n-22 


Participating 

Schools 

n=22 


Total 

n=54 


Percent 

% 


Increase levels of self- 
esteem and confidence 


10 


20 


14 


44 


81% 


Provide career-related 
employment experience 


9 


19 


12 


40 


74% 


Expose youths to new and 
innovative technologies 


10 


8 


13 


31 


57% 


Help youths with the 
transition from school to 
work 


9 


9 


13 


31 


57% 



Case studies also indicate that to date many respondents believe the strategy’s objectives are 
being met. For example, summer positions were created, cooperative education programs have 
been developed or expanded, science camps have been accessed or developed, and links with the 
community and employers established Focus groups with students in Manitoba and 
Saskatchewan also indicate that many students believe the strategy programs encourage them to 
stay in school, build self-esteem and motivate them to continue their studies. Respondents from 
the case studies also acknowledge that it is too early to assess the impact on graduation rates and 
academic marks. 



Communication 

Most regions communicated information about the strategy by sending letters to Chief and 
Council, and Directors of Education. Of the interviews conducted randomly with First Nations 
and Inuit, over 65% of respondents participated and said they were aware of the strategy’s 
summer employment program. Slightly fewer respondents were aware of the science camp 
program and only one-third said they were aware of the Cooperative Education Program. 

We found that several eligible First Nations believed they did not have the resources to develop a 
cooperative program. Many respondents expressed an interest in communicating with First 
Nations who did have a program. While DIAND headquarters prepared a manual explaining how 
to set up a cooperative education program, regions did not widely distribute it to First Nations. 
Some regions stated the manual was too cumbersome. Most thought it appropriate for First 
Nations organizations to explain the program to communities. 



30 
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Delivery Mechanisms 

Most regions opted to transfer part or all of the strategy administration to First Nations and Inuit 
organizations. Those that did not, indicated they could not find an Aboriginal organization willing 
to take the responsibility on such a short time frame. 

Regional personnel interviewed were concerned that the administration of the strategy placed an 
additional burden on them, especially with downsizing and extra responsibilities. Respondents 
stated that there is little time to review proposals and progress reports. They stated that they 
must devote more time to do this work than other programs of substantially greater cost. All 
respondents agreed that the most effective way to deliver the strategy was through one or more 
regional First Nations and Inuit organizations. For example, in Quebec, five First Nations 
organizations administer the strategy on behalf of the diverse communities in that large region. In 
Saskatchewan, the FSIN manages the strategy on behalf of all First Nations in the region. 

Through case studies and interviews with First Nations and Inuit, we found that regionally based 
First Nations and Inuit organizations (Refer to Model 1) appear to be most effective for the 
following reasons: 

• First Nations and Inuit organizations have expertise in youth programming. 

• existing infrastructure, networks and contacts with First Nations and Inuit communities 
are used to communicate and distribute information, and to share experiences. 

• organizations provide help and advice to develop proposals. 

• First Nations and Inuit organizations are involved in other activities such as recruiting 
national employers, for example, banks, airlines or utility companies. 

• First Nations and Inuit are more willing to report information to their own organizations 
because they feel a greater sense of ownership over programs. 

Some First Nations and Inuit also have a mandate in education which allows them to provide 
expertise and advice in developing curriculum for a cooperative education program. 



n •« 
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General Indicators of Cost-Effectiveness 

As stated, regions where the strategy is administered by First Nations and Inuit organizations 
appear to be more effective in implementing and delivering the youth programs. Several 
indicators suggest this approach may also be the most cost-effective. The Statistical Overview 
identifies the following indicators: 

• overall percentage spent on managing the strategy; 

• leverage ratio (i.e., funding from other sources); 

• percentage of First Nations reporting on programs; and 

• the percentage of funds that went directly to youth. 

These indicators focus on the Summer Student Career Placement Program for which data are 
most complete. Data are less complete for the Science and Technology Camps Program and as of 
yet unavailable for the Cooperative Education Program. 

Table 9 provides a regional comparison of the above mentioned indicators for the Summer 
Student Career Placement Program. The Departmental Audit and Evaluation Branch utilized the 
data from the program evaluation forms submitted by First Nations and Inuit. The data were 
filtered to remove incomplete information and inconsistencies in reporting. 

• The leverage ratio is the percentage of additional funds contributed to the strategy for 
every dollar DIAND provides First Nationsand Inuit. 

• The reporting indicator is the percentage of reported expenditures versus the federal 
portion allocated by DIAND. 

• The percentage spent on managing the program and the percentage directed to students 
are self-reported data from program evaluation forms submitted by First Nations and Inuit 
and their organizations. 
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Table 9 



Data from Sample of First Nations and Related Organizations 
Summer Student Career Placement Program 






Self-reported data from evaluation forms 


Region 


Leverage Ratio 
(Other 

Contributions) 


Percentage Spent 
Managing Program 


Percentage 
Directly to 
Students 


Reporting 

Indicator 


Atlantic 


0.49 


11% 


89% 


100% 


Quebec" 


0.80 


6.8% 


93% 


100% 


Ontario 


0.43 


8.1% 


86% 


74% 


Manitoba 


0.22 


9.6% 


91% 


93% 


Saskatchewan 


N/A 


7.0% 


93% 


100% 


Alberta 


0.86 


13% 


86% 


83% 


British Columbia 


0.33 


8.4% 


84% 


20% 


Yukon 


0.19 


5% 


95% 


51% 


Northwest Territories 


0.40 


13% 


82% 


44% 


Total 


0.43 


9.3% 


89% 


74% 


Source: Departmental Audit and Evaluation Branch, Statistical Overview, June 1997. 


Definitions 

• Leverage Ratio: The rate comparing additional funds from other sources 
contributed by First Nations to funding provided by DIAND. 

• Reporting Indicator: Reported total expenditures by First Nations and Inuit versus 
the federal portion allocated by DIAND. 



In Saskatchewan, and Quebec, where the regional administration of the strategy was fully 
transferred to First Nations organizations, program reporting for the summer programs 
was 100%. Other regions which did not fully transfer regional activities reported lower 
return rates on program reporting, such as British Columbia with 20% and the Northwest 
Territories with 44%. The lower rates mean that reporting is incomplete and program 
results are greatly under represented in some regions. 



Figures do not take into account an additional $71,500 the Quebec Region 
provided to First Nations organizations to administer the strategy. This funding 
came from the region’s own budget. 

O o 
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• The sample of First Nations and Inuit reported that 9.3% of total funds was spent on 
managing the program 11 . This compares favourably to other evaluations of economic 
development initiatives such as the Atlantic Canada Operating Agency (ACOA) and the 
Canada/Territorial Economic Development Agreements which found higher percentages 
spent on managing the program 13 . 

• The case studies indicate that First Nations organizations in Quebec and Saskatchewan 
have been effective in managing strategy programs. As seen in Table 9, Quebec 14 and 
Saskatchewan First Nations reported spending 6.8% and 7.0% of the funds to manage the 
program. These are among the lowest rates of expenditures on administration 15 . 

• Program reporting data indicate that 89% of the total funds spent went directly to First 
Nations and Inuit youth. This means that 1 1% was spent on other activities, such as 
management compared to the 9.3% that was reported by First Nations and Inuit as being 
spent on program administration. However, the statistical overview found that the 
difference is explained by under-reporting and because many First Nations and Inuit had a 
different understanding of the definition of “percentage spent on managing”. 

• The total leverage ratio for First Nations and Inuit is 0.43. This means that for every 
dollar allocated by DIAND to First Nations, they contribute just under half that amount in 
additional funding. In 1996-1997, First Nations and Inuit reported total expenditures of 
$6.3 million on the Summer Student Career Placement Program as follows: 

$4.4 million from DIAND funds 

$1.9 million contributed by First Nations and Inuit ($1.9 million represents 43% of 

DIAND’s $4.4 million allocation or a leverage ratio of 0.43). 

This is an indication of First Nations and Inuit willingness to integrate the program into 
their communities’ overall economic plans. 



At present we do not have information on how many First Nations did not report 
on the program. 

Departmental Audit and Evaluation Branch, Statistical Overview, June 1997. 

Our file review found that Quebec provided First Nations organizations an 
additional $71,500 to administer programs; therefore, its percentage spent on 
management is higher than reported. 

The Yukon reported a rate of 5% of funds spent on managing the program. 
However, only half of the communities reported to date. 

34 
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Complete and consistent reporting of these ongoing results measures will assist the department in 
the future to determine the outcomes of the five strategy programs. We discuss these ongoing 
results indicators and others in the evaluation framework that accompanies this evaluation study 



Reporting and Accountability 

In terms of accountability, DIAND headquarters and regional staff were satisfied that the strategy 
was well managed and First Nations and Inuit were accountable for spending the program dollars 
for the purpose specified in the contribution agreement. However, a few respondents commented 
that this is not in the spirit of the new Financial Transfer Agreement and self-government. 
Managing the strategy in this way is not a problem in the short term, but several First Nations and 
Inuit stated that youth programs should become part of their mandate in education and that they 
should have freedom to allocate funds and design programs accordingly. 



In terms of reporting, we identified two problems: under-reporting and the quality of reporting. 



Under-reporting: While headquarters respondents are satisfied that funding is spent 
appropriately at the local level, they are having difficulty obtaining roll up data from a few 
regions, resulting in under-reporting on program outputs on the Summer Student Career 
Placement and Science and Technology Summer Camp programs. In addition, the 
Statistical Overview found that regions and headquarters reported on $4,925,025 of the 
$6,000,000 set aside for the Summer Student Career Placement Program. Therefore, at 
present there is under-reporting of approximately $1,000,000. It will be the role of the 
financial audit submitted by each First Nation and Inuit organization as part of the terms 
and conditions of their funding agreement to determine how funds were expended. 

Quality of reporting: The quality of reports is also an issue identified in the Statistical 
Overview which found that First Nations and Inuit organization used different definitions 
for reporting items such as the amount spent on managing the Summer Student Career 
Placement Program. In addition, our initial file review found that reporting on the 
Cooperative Education Program has not been uniform, with some First Nations and Inuit 
completing the evaluation forms, and others submitting more detailed reports on their 
progress. In the latter case, over-reporting too can lead to inefficiencies. 
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Partnerships 

The theme of the strategy is one of partnerships at various levels 

• First Nations and DIAND: This partnership consists of DIAND funding the strategy and 
providing some support and information at the regional and headquarters levels. 

• Private Sector: Overall, most respondents stated that partnerships with private sector 
firms off-reserve are developing. In addition. First Nations organizations are taking an 
active role in recruiting large corporations such as banks and utility companies. Most of 
the partnerships with the private sector are on-reserve and range from the local gas bar to 
logging firms. 

• Other Government Departments and Levels of Government: These partnerships 
consist of youth programs that provide summer employment or other job placement. First 
Nations and Inuit identified several federal departments involved in youth programming, 
including: 

HRDC’s Youth Services Canada, Summer Career Placement, Pathways to 
Success, and Native Internship programs; 

Industry Canada’s Student Connection and SchoolNet programs; and 
Health Canada’s Brighter Futures program. 

Provincial ministries did not play a large role but First Nations did identify placements 
with school boards, ministries of education, social services, and health. 

• First Nations Governments and Agencies: First Nations and Inuit identified 
partnerships among regional First Nations organizations, education authorities, school 
boards. Tribal Councils, Chief and Council, and economic development corporations as 
the most important. There is a philosophy that youth are an integral part of the 
community’s social and economic future. Therefore, youth programs are a priority 
incorporated into other community activities. Communities rely heavily on all employers 
on-reserve, from the health unit, band council office, or the local restaurant. First Nations 
and Inuit not participating in the strategy mentioned the use of other youth programs such 
as those for substance abuse and recreational programs. 

Interviews with First Nations respondents and case studies indicate that partnerships developed at 
the community level will endure even without strategy funding. First Nations and Inuit have well 
established partnerships with other departments, such as Human Resources Development Canada. 
For example. First Nations have been involved in the Pathways to Success Aboriginal local area 
management boards for many years, and in some cases these structures have also taken on 
responsibility for HRDC’s programs geared to Aboriginal urban youth. First Nations are in the 
process of developing relationships with the private sector off-reserve, so it is too early to assess 
the sustainability of these partnerships. 
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Strengths and Weaknesses 

Table 10 identifies the most commonly mentioned strengths and weaknesses. 



Table 10 





Strengths and Weaknesses of the Initial Strategy 




Strengths 


Weaknesses 


Objectives 


The strategy’s objectives complement 
DlAND’s and communities’ 
priorities for enhancing economic 
development on-reserve and in lnuit 
communities. Therefore, the 
objectives are viewed as appropriate 
and relevant. 

The strategy provides valuable work 
experience that will benefit in-school 
youth with future work practices and 
their careers. 


Some respondents state that larger communities with 
resources and many students will benefit more from the 
Cooperative Education Program, as opposed to small 
remote communities with no coordinators and limited 
placement opportunities. 

Respondents state that there is a fundamental 
difference between post-secondary and high school 
cooperative education programs. Many believe 
expectations for the program are too high because 
secondary' school placements may not lead to career 
choices. 


Guidelines 


Most First Nations and lnuit believe 
that the guidelines are flexible as they 
adapted them to their needs. 

However, other communities perceive 
them to be restrictive but often they 
do not attempt to adapt them to their 
needs. 


Several First Nations and lnuit expressed concern with 
the fact that the Strategy did not provide funding to 
deal with issues such as the payment for Workers 
Compensation benefits which are required for students 
who work off-reserve. This is viewed as important for 
new partnerships with the private sector and expanding 
job opportunities available to students. 

The Summer Student Career Placement Program did 
not cover the additional costs of travelling long 
distances to take a job placement off-reserve. This is of 
particular concern to remote and northern communities 
that send students to other communities or urban 
centres for work experience. 


Reporting 


Most First Nations and lnuit, their 
organizations, and DIAND staff felt 
that the current level of reporting was 
adequate given the amount of money 
expended on the strategy. 


There is under-reporting which results in an 
incomplete picture of program outputs and outcomes. 
Under-reporting is most prevalent in British Columbia, 
the Northwest Territories, and the Yukon. 

The quality of reporting varies among regions. There 
are some differences in the understanding of the 
definition of items reported on evaluation forms, such 
as the percentage spent on managing. 
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Strengths and W eaknesses of the Initial Strategy 




Strengths 


Weaknesses 


Partnerships 


Programs are based on local and 
regional partnerships between 
Directors of Education, education 
authorities. First Nations and Inuit 
governments, program coordinators, 
schools, business, the community and 
students. 

The strategy is a community* and 
youth-driven initiative that provides 
exposure to new experiences and 
learning activities 


Several First Nations and Inuit respondents stated that 
often there are limited opportunities on-reserve and few 
chances to develop partnerships with private 
employers. 


Communication 


Overall, First Nau./.s and Inuit were 
aware of the strategy and its 
programs. 


Many First Nations reported that they wanted to have 
more information on the Cooperative Education 
Program, preferably in a forum where they could 
exchange ideas with schools that had successfully 
implemented the program. 


Timing 


DIAND and First Nations and Inuit 
organizations acted quickly to 
disseminate program funds once the 
strategy was announced late in the 
school year. 


The timing of the strategy led to delays and frustration, 
especially with the Cooperative Education and Science 
and Technology Camp programs. In some cases. First 
Nations and Inuit were unable to submit proposals for 
science camps because they received the information 
too late. The renewed strategy w*as announced earlier 
in the school year. 


Resources 


The strategy funds have been 
increased from $10 million the first 
year to $25 million annually for the 
next two years. 

First Nations and Inuit are able to 
lever significant funds from other 
sources to contribute to the Summer 
Student Career Placement Program. 


Several First Nations and Inuit communities 
commented that resources were limited. For example, 
for a First Nations to establish a cooperative education 
program requires workbooks and other tools but no 
money is available for these items. 

DIAND regional coordinators stated that there was 
minimal lime and departmental resources available to 
administer the strategy in-house. 
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Section 5 - Measuring Impacts 



This section will discuss the short-term impacts of the strategy. It will take several years to 
observe long-term impacts. We offer suggestions for what to measure and how to collect data 
later in this section and in the evaluation framework accompanying this report. 



Initial Short-Term Impacts 

We used both qualitative (interviews, case studies, focus groups) and quantitative (statistical 

overview) approaches to assess short-term program results. Most of the data are based on 

respondents’ perceptions of impacts to date. 

Intended Impacts 

The evaluation study found short-term results in several areas. 

• Creation of Career-Related Summer Employment: The Statistical Overview found 
that 3,768 students participated in the Summer Student Career Placement Program. The 
majority of the students worked for 4 to 6 weeks during the summer. Interviews with 
First Nations and Inuit stakeholders indicate that there were a wide range of placements 
created. The following sectors were identified most often: 

service industry (i.e., retail, restaurant, gas bars, etc.) 
administration (i.e., band office. Tribal Council, etc.) 
health, social services and education 
construction and mechanical 
recreation 

economic development organizations 

Case studies and interviews also indicate that often communities hire students to 
coordinate other youth programs, such as the Summer Student Career Placement 
Program, and the Science and Technology Camps Program. 

• Acquisition of Job Skills: In focus groups, youth who participated in the Summer 
Student Career Placement and Cooperative Education programs reported positive 
experiences. Table 1 1 lists some of their quotes. Students stated they acquired valuable 
skills to prepare them for future jobs, including: 

skills to compete for and acquire a job, such as preparing resumes, cover letters 
and conducting interviews; 
administration and computer skills; and 
interpersonal and communication skills. 
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Table 11 



Quotes from students who participated in focus groups 

"1 'm working with children in kindergarten and 1 enjoy it. It 
gives me confidence and 1 feel good about myself 1 am now 
going to pursue a teaching degree. " 

"This placement introduced me to a job and work. It showed 
me haw to work with the RCMP. ” 

“The program showed me how to have a job and what to do. 
Our time in school taught me about job interviews, cover letters 
and resumes. ” 

"We are taught real world skills. ” 



Exposure to Science and Technology: The Statistical Overview found that 
2,032 students participated in the Science and Technology Camps. File review in three 
DIAND regions reveal that many communities opted to access computer camps at nearby 
universities or to develop camps that often focussed on traditional plants and medicines. 
Most of the participants were between 12 and 13 years old. Coordinators in the three case 
study communities believe that many students would otherwise not have been exposed to 
the sciences. Coordinators also believe the positive experience of the younger students 
will generate demand for a follow-up camp at the junior and high school levels. 

An organization in one of our case studies developed a mobile camp program that exposed 
science to communities throughout the province. 

Development and Expansion of Existing Cooperative Program: The Statistical 
Overview found that to date available information shows that 879 students and 
1 14 schools participated in the Cooperative Education Program. 

We conducted interviews with 25 participating schools and found that 36% of the schools 
had started programs in the fall of 1996 or January 1997, and another 28% expanded 
existing programs. Other schools were in the planning and development stages. 

Interview findings with participating First Nations schools also indicate that the majority 
of respondents believe the quality of placement positions is either good or excellent. 
Interviews and case studies also identify two streams of cooperative education: one stream 
is geared to special needs students and focuses on life skills; the other stream is available 
to all regular high school students. 
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The three case studies reveal that the cooperative programs are diverse. One community 
used the program to help launch a micro-business venture while another sought 
placements in urban centres where their students will eventually pursue post-secondary 
studies. 

The Development of Partnerships . All stakeholder groups, including those that did not 
participate in the strategy, reported partnerships with other parties involved in Aboriginal 
youth programming. Table 12 identifies the partners most often mentioned 



Table 12 



Table 12: Findings of Interviews with First Nations and Inuit Stakehold 
Parties Reported to be Involved in Aboriginal Youth Programming (n=9 


ers 

•5) 




Participating 

FNs 

(n=26) 


Non- 

Participating 

FNs 

(n=28) 


FN 

Implementing 

Organizations 

(n=10) 


FN 

Coordinators 

(n=31) 


Total 

% 


Local partnerships at First Nations 
level 


13 


7 


0 


23 


45% 


Human Resources Development 
Canada 


3 


7 


7 


18 


37% 


Provincial Ministries 


3 


4 


3 


17 


28% 


Health Canada 


2 


5 


1 


3 


12% 


Industry Canada 


2 


2 


2 


3 


9% 


Private Sector 


N/A 


N/A 


2 


2 


4% 


First Nations Interviewed that Participated in the Strategy’s Programs (n=67) 


DIAND Programs 


Summer Placement 


0 


N/A 


6 


20 


39% 


Science &. Technology Camps 


0 


N/A 


9 


9 


27% 


Cooperative Education 


22 


N/A 


3 


4 


43% 


Note: Respondents from Participating First Nations schools did not identify the private sector as a partner but 

they did refer to placements with private employers such as retailers. Non-participating schools did 
not identify private sector partners. 
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• Positive Influence on Students: Young people in focus groups in Manitoba and 

Saskatchewan stated that strategy programs had positively influenced them, in particular 
they mentioned: 

an increase in self-esteem and confidence 
encouragement to stay in school 
enhanced interest in their studies 
increased sense of responsibility 

ability to better relate the school experience to employment 
knowledge of the types of jobs that exist, especially in the community 

Cooperative Education coordinators in the case study communities also believe there are 
other benefits: 

an increase in class attendance 

some drop-outs are encouraged to return to school 

teachers and coordinators are better able to engage in direct career planning with 
students 

greater identification of students with professional career aspirations. 

Respondents also made a number of observations about each of the strategy programs described 
in Table 13. 
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Observations of Strategy Programs 
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Unintended Impacts 



Respondents noted a few unintended but positive impacts associated with the strategy. 

• Facilitate the transition from a rural to an urban setting: Respondents from one case 
study community stated that the Summer Student Career Placement and Cooperative 
Education programs help students make the transition from a rural to an urban setting. 
Respondents stated that many students go away to university but fail because they cannot 
find jobs to help support them during school. The programs try to make the students 
better able to compete in an urban environment. 

• Creation of a micro-business: One case study community created a micro-business (i.e., 
catering) using students. Cooperative education students with special needs are trained to 
work in the business. It is expected that the program will create full-time employment as 
the community is planning to hire the students after graduation. 



Long-Term Impacts 

It is too early to assess whether long-term impacts will be achieved. It will take years to know if 
there has been an increase in high school graduation rates, long-term employment opportunities, 
and the number of Aboriginal First Nations and Inuit youth choosing a career in science or 
technology. Most respondents believe that the strategy’s goals are reasonable and impacts will 
eventually occur. However, many respondents recognize that it will be difficult to measure long- 
term impacts and to attribute results to the influence of the strategy. 

Part of the evaluation study exercise was to develop an evaluation framework that identifies 
program activities, outputs, outcomes and short and long term impacts. It is the role of an 
evaluation framework to provide suggestions on how to assess long-term impacts. The 
framework also discusses performance measurement and collection strategies. 



Performance Measurement Issues and Future Evaluation 

This section summarizes the information needs identified by stakeholders and current performance 
measurement activities being undertaken. It also includes an examination of the short-term, 
ongoing data requirements currently being collected and ways in which this data can be 
strengthened. In addition, this section outlines potential long-term performance indicators and 
associated data collection approaches to the future evaluation which will involve a partnership 
approach between the department and First Nations and Inuit and related organizations. 
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Performance Information Utilization 



The evaluation found that information needs vary by stakeholder group. The case studies 
undertaken as part of the evaluation found that at the local First Nation and Inuit level, 
information needs focus on day-to-day information which measures the operation of the strategy - 
- what is done, how it is done and what it costs. In addition to reports to DIAND which focus on 
key short-term outputs, outcomes and administrative efficiency. First Nations and Inuit 
parti cpating in the case studies also collect information on: the number and type of employers; 
skills acquired by the students; student and employer satisfaction; steps involved in the selection, 
recruitment and matching processes; job preparation activities; and resources devoted to the 
programs. 



Figure 10 presents an overview of the information needs of the various strategy stakeholders. 
Generally, the evaluation found 
that performance information 
serves operational and strategic 
purposes by: 

• Supporting First Nation 
and Inuit and departmental 
decision-making to 
improve activities such as 
program design, criteria, 
implementation 
mechanisms and 
accounting for 
performance; 

• Ensuring that programs 
reflect the needs of 
students and employers; 

• Ensuring that First Nations 
and DIAND resources are 
properly targeted to 
achieve best program 
outcomes; and 

• Ensuring that the rationale 
for the programs are 
properly documented to 
support future resource 



Figure 10: Overview of 
Information Needs by 
Stakeholder Group and Level 
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- ongoing results 
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impacts 
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The case studies undertaken as pan of the evaluation underline the importance of performance 
information for First Nations and lnuit implementing the strategy. For example, one case study 
found that for the Cooperative Education Program, ongoing employer and student evaluations 
are used to monitor and adjust program implementation to ensure that students’ needs are met. In 
this case, if a student is not acquiring the skills, the coordinator will meet with the employer and 
changes to the placement are made. If an employer does not keep students challenged, the 
program will not send others there. The coordinator reported that this had only happened once 
and that employers tend to be very cooperative. 

Ongoing Data Collection Activities 

Interviews with regional D1AND representatives found that the current ongoing reporting 
requirements meet their needs to account for funding and are felt to provide sufficient information 
about short term outcomes. The case studies undertaken as part of the evaluation found that the 
ongoing information collected by First Nations for their own management purposes exceeded the 
departmental information requirements. 

The interim evaluation found a variety of measurement activities occurring at the local First 
Nation and lnuit level and regional implementing organizations. For example, First Nations and 
regional implementing organizations interviewed as part of the evaluation noted that they are 
collecting demographic information on participants in the strategy. Some organizations also 
gather information from students and employers about their experiences with the strategy. These 
organizations said they talk to the students and employers themselves, or they have the students 
fill out an evaluation of the program. 

In addition, representatives of First Nation schools participating in the strategy’s Cooperative 
Education Program noted that the program reporting requirements include: progress reports, 
interim reports, and a final report. The types of information collected on students include, 
interviews with students, employers and teachers; student diaries; pre and post placement 
evaluations; on-site visits, and; employer evaluations. 

A review of the short-term information requirements needed to assess program outcomes was 
undertaken as part of the development of the evaluation framework prepared as part of this 
evaluation. While this review confirms that the short-term performance information requested by 
the department is sufficient for short-term accounting for funding, our study identifies a number of 
additional key information elements that should be collected to ensure that program objectives can 
be accurately assessed. For example, to ensure that the strategy’s objective of promoting career 
related skills can be assessed, ongoing program reporting should be modified to include 
information on the type of jobs created. Table 13, below, provides an overview of the 
information currently being collected and the additional data elements which should be requested 
by the department. The evaluation framework (Appendix 6) provides more detail on current 
information requirements and the findings of this review. 
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Table 14: Examination of Ongoing Departmental Data Requirements 


Summary of Current Departmental 
Reporting Requirements 


Areas Identified for Inclusion 


✓ Outputs: Projects funded 

✓ Outcomes: Jobs & placements created; 
duration of placements; age and grade 
of participants; average wages 
received by participants 

✓ Administrative Efficiency & 
Partnerships: funds levered (i.e., 
contributions from other sources) and 
percentage of funds spent on 
managing programs 


✓ Type of employer by category 
(i.e., band, federal, provincial, 
private) 

✓ Type of placement by sector (i.e., 
administration, health, education, 
etc.) 

✓ Better definition of information on 
other funding sources contributing 
to the program (i.e., band revenue, 
federal departments, etc.) 

✓ National tracking of First Nations 
and Inuit participating in Science 
and Technology and Summer 
Student Career programs 



Long-Term Information Requirements 

Although the review of current reporting requirements found that the short-term information 
requirements are appropriate, the evaluation identified a lack of performance information to assess 
the long-term, intended impacts such as promoting increased graduation rates and facilitating the 
transition from school to work. 

Most First Nation respondents stated that tracking students over the years would be an effective 
way to measure performance, especially in the Cooperative Education Program. However, 
interviewees noted that at this early stage it is not clear what type of information they should 
collect nor what DIAND’s future needs will be First Nations and Inuit respondent suggestions 
for indicators include: drop-out / graduation rates; absentee rates; careers selected, withdrawals 
from programs, and changes in students’ academic marks, self-esteem, and the transition from 
school to work. All respondents of the regional implementing organizations stated that the best 
way to measure the impacts of the strategy is to ask students and employers to evaluate the 
strategy. However, First Nations respondents acknowledged that it would be difficult to assess 
whether or not program performance was due to the strategy. Those we interviewed cautioned 
that it would be difficult to attribute changes in the indicators to the strategy as they are also 
influenced by other factors such as social conditions and lack of employment opportunities. 

Figure 1 1 presents a summary of potential performance indicators to assess long-term impacts and 
associated data collection methods. More detailed information is contained in the evaluation 
framework. 
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Suggested Approach to the Long-Term Evaluation 

Although the interviews with the First Nations and Inuit Implementing Organizations and local 
coordinators found that currently they have no plans to undertake a long-term evaluation, they 
expressed a high level of interest in doing so. At present, their focus is on enhancing Strategy 
Programs but respondents were interested in measuring long term impacts and felt that a First 
Nation directed, regional approach is best suited to assess long-term impacts. Regional 
departmental officials were in agreement that First Nations and their regional implementing 
organizations should take the lead in evaluating long term impacts. 

The evaluation identified the following principles for collecting long-term performance 
indicators and for undertaking the future evaluation: 

• First Nations and Inuit and DIAND need to work together to define their information 
needs; 

• A performance framework should reflect local needs. First Nations and Inuit partners are 
managing programs and require information to ensure that programs reflect the needs of 
students and employers; 

• Performance monitoring should be relevant to First Nations and Inuit and departmental 
local, regional and national information needs, as well as reflect those of central agencies; 
and 

• The list of indicators should be concise, reflecting the size of the initial investment in 
programming and not be overly expensive or burdensome. 
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FIGURE 11 : KEY LONG-TERM PERFORMANCE INDICATORS 
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The evaluation framework (Appendix 6) identifies an approach for collecting the long-term 
performance information and for the conduct of the future evaluation. This approach involves 
First Nations and Inuit and DIAND working together on the development of evaluation tools to 
support the collection of information to track the long-term impacts of the strategy. Figure 12 
presents an overview of this proposed approach which involves working with First Nations and 
Inuit to develop the necessary evaluation tools such as local self-evaluation guides and First 
Nations and related organizations involved in their implementation. 



Figure 12: Proposed Approach 
for Future DIAND Youth Strategy Evaluations 
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Section 6 - Lessons Learned and Best Practices 



There are many lessons learned and best practices to share with other communities described 
below. 



Implementation of the Strategy 

First Nations Organizations: In case studies, we observed that in regions where First Nations 
organizations administered programs, the strategy was implemented more quickly as existing 
networks and structures were used to communicate the objectives of the strategy and provide 
direction on how to set up a program. First Nations organizations were better able to address 
concerns on a regional level while in other regions individual communities had to deal with 
problems on their own. 

Best Practice Example 

One region provided additional funds to First Nations organizations to implement and administer 
the strategy. The region realized that initial, start-up resources were required for administrative 
assistance to transfer responsibilities in a short time frame. 



Communication 

Communicating Information in a Useful Way: First Nations and Inuit prefer using existing 
structures, such as education committees, conferences, workshops, or First Nations and Inuit 
organizations to learn about programs. While D1AND has provided information, such as the 
program guidelines and a manual on cooperative education, they are not the most effective as a 
communication tool. Many respondents noted that it is easier to learn from each other, especially 
from communities that have successfully implemented the strategy 

Best Practice Example 

A First Nations organization in one region held a workshop with community representatives to 
explain the strategy, how to apply for programs, and how to set up programs. Regular follow-up 
teleconferences have been held to share experiences and to work out common problems. 

Best Practice Example 

One First Nation organization sends out a newsletter via facsimile with information on current 
activities in communities. The First Nation organization commented that other communities have 
contacted a featured community (in the newsletter) to request if their students might participate in 
a program or activity. 
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Program Effectiveness 



Recruiting Students and Employers: First Nations and Inuit stakeholders report considerable 
success recruiting employers and students. Communities recruit students through advertisements 
in newspapers, notices in malls, newsletters, school assemblies, or other announcements. 
Respondents report that there is no shortage of students applying for the Summer Student Career 
Placement and Cooperative Education programs. 

Coordination at the Local Level: We observed through interviews and case studies that in 
several communities, coordinators for the Summer Student Career Placement and Cooperative 
Education programs work closely together. The coordinators indicated that they share 
information, work together on committees, and share a common pool of employers. This 
interaction is believed to benefit students and many end up participating in both programs. 

Best Practice Example 

One First Nations organization encourages several First Nations to work together to share 
expertise. For example, three British Columbia communities are working together on the Summer 
Student Career Placement Program by sharing placements. 

Summer Student Career Placement Program: First Nations and Inuit respondents reported 
providing a broad range of placements to students The Statistical Overview also shows that most 
communities offer 4 to 8 week placements which respondents believe allow more students the 
opportunity to obtain a job placement. 

Science and Technology Camps Program: The Statistical Overview indicates that science 
camps are geared toward students between 12 and 13 years of age which is different from the 
original guidelines which target students from 12 to 20 years. Some First Nations and Inuit 
respondents indicated that they had younger children participating in camps. Case study 
respondents believe that as interest grows, additional camps geared to older students will be 
needed. 

Through case studies and a review of program progress reports, we observed that successful 
camps were those that made use of existing resources, such as university computer labs, or those 
that contained a traditional component. 

Best Practice Example 

One First Nation organization encourages communities to access, whenever possible, “ready- 
made” programs from universities or other agencies. This First Nation organization reports 
keeping an inventory of existing camp programs and resources around the province and refers 
First Nations to them. The organization also reports contacting resource-based agencies to 
develop a new camp or include activities in First Nations camps (e.g., visit to a hydro dam). 
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Best Practice Example 

Many First Nations organizations have put a cap on spending for the camps and encourage 
communities to match the funds. 

Best Practice Example 

One First Nation organization developed a mobile science camp program that allows many 
communities across the province the opportunity to expose students to science. The program has 
developed a curriculum for First Nations students and also provides professional development and 
training opportunities for teachers in local schools. 

Cooperative Education Program: In our case studies, we observed that successful cooperative 
education programs have a number of common features: 



• a full-time coordinator to work with students and recruit employers; 

• well developed curriculum and classroom work as well as job placements; 

. manuals and other communication materials to explain program objectives to students, 
parents, teachers, and employers; 

• on and off-reserve partnerships with the private sector and government agencies providing 
a wide range of jobs and potential career opportunities; 

• extensive planning and preparation is put into developing the program; 

• support in the community, usually a resolution from the Chief and Council or school 

board; and 

• coordinators work closely with their counterparts administering the Summer Student 
Career Placement and Science and Technology Summer Camp programs to combine 
efforts. 

In our case studies, we observed that if the above conditions are present, both streams of 
programming, one geared to regular students and the other to those with special needs, work 

well. 

Best Practice Example 

Because of the limited funding, one First Nation organization decided to focus the Cooperative 
Education Program on schools that can provide resources and employment opportunities for 
students. Only schools with at least 30 participants and established on and off-reserve 
partnerships were selected for the program. 
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Community Support 



Political Support: First Nations respondents reported that the support of Chief and Council is an 
integral part of success. Often the Chief, Council, and education authorities are also actively 
involved in promoting their community and recruiting employers. 

Best Practice Example 

In one community, all political leaders and agencies are involved in promoting its programs and 
recruiting employers to participate in the Summer Student Career Placement and Cooperative 
Education programs. Chief and Council actively seek out partners from all sectors on and off- 
reserve that interact with this First Nation organization. 

Community Involvement: First Nations respondents report that integrating the strategy’s 
objectives into the community’s economic plans and activities is also a factor in program success. 
Respondents also believe community support is important to help students address problems. 

Best Practice Example 

One regional First Nation organization noted a problem with “spring drop-off.” Many adult 
students have family responsibilities and find it a challenge to stay in school or travel to a job 
placement. Program coordinators have made arrangements to help students with these 
obligations by having them work one day a week or to travel one week out of the month to gain 
their work experience. The First Nation organization commented that “drop-off” has been 
reduced. 



Expansion of Existing Programs 

Building on Existing Programs: In interviews, we found that several communities used 
program funds to expand existing youth programs, in particular summer employment and 
cooperative education programs. 



Partnerships 

On and OfT-Reserve Partners: Both DIAND and First Nations and Inuit respondents stated that 
on and off-reserve partnerships are the key to providing a broad range of summer and cooperative 
job placements. Off-reserve placements expand upon opportunities available on-reserve. 

Leveraging Funds: The Statistical Overview found that on average communities leverage 28% 
of total spending on the strategy. In one case study, we found a community where 85% of funds 
spent came from band revenues. 
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Access to Other Federal Programs First Nations and Inuit interviews indicated that DIAND 
programs are only one source of youth programming for communities. We interviewed First 
Nations not participating in the strategy who also reported accessing HRDC programs such as the 
Summer Employment Development Program, the Native Internship Program, and Youth Service 

Canada. 



Reporting 

Incentives to Report: DIAND headquarters and regional staff reported some problems with 
obtaining reporting information. Some respondents believe incentives, whether positive or 
negative, may be needed to obtain reporting information. 

Best Practice Example 

One First Nation organization holds back 10% of program funds that are allocated to member 
communities when they submit their evaluation forms. We also found that the First Nation 
organization included information about the importance of reporting in its program information 
booklets sent out to members. Respondents reported that this method works well. 

Best Practice Example 

One DIAND region reported holding back 50% of program funds until First Nations submitted 
their evaluation forms. The region indicated that this has been found to work effectively. 
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Section 7 - Suggested Improvements 



Strategy stakeholders identified many suggestions for improvements to the strategy and its 
programs. Some are directed at the department while others are what First Nations and Inuit 
would like to improve at the local level. 



DIAND 

DIAND and First Nations and Inuit respondents made the following suggestions for how DIAND 

could improve the strategy. 

• The original strategy did not deal with the many out-of-school and unemployed youth, 
especially those in remote communities. Respondents stated that there should be a 
program for out-of-school and unemployed youth. They suggested that the strategy 
should encourage students to create new business employment opportunities. 

• DIAND should distribute information about programs as soon as possible. This will help 
First Nations who need more time to consult and develop programs. 

. Regions would like to transfer activities such as regional coordination and administration 
of the strategy to First Nations regional organizations, but in some cases organizations are 
reluctant to take the responsibility. Some regional respondents stated that the strategy 
should include a separate amount for First Nations organizations to administer the 
program. Respondents suggested the funds could be a percentage of the overall strategy 
funds eliminating the need for separate accounting. 

• DIAND regional staffs role should consist of ongoing contact with communities and 
progress reviews. Regional staff state that the focus should be on monitoring, not 
program administration. 

• Regional respondents suggested shorter guidelines for each program to facilitate 
communication and allow First Nations flexibility. Since First Nations design their own 
programs, they believe a lengthy policy is not needed. 

• Several First Nations respondents believe the guidelines of the summer camps should be 
changed to allow younger children to participate in the program. 

• Regional respondents indicated that DIAND should help First Nations establish a forum to 
share best practices and other experiences. Respondents also suggested DIAND play a 
role in informing communities about other federal programs available to them, such as 
those offered by Industry Canada and HRDC. 
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First Nations 



First Nations respondents identified several areas to improve programs. 

• First Nations need to develop more jobs in students’ career related fields of interest and 
should share placements among communities if necessary. 

• The student selection and matching process needs to be improved to permit students to 
gain employment opportunities in their field of study. In some communities, placements 
are limited or are given out on a rotational basis so some students may not have the 
opportunity to pursue their preferred job. 

• First Nations suggested their elders be included in the implementation and delivery 
process. Elders can teach about traditional ways and link the students to other initiatives 
in the community. 

• Some respondents suggested including the development of an anti-racism strategy for the 
workplace into First Nations communities programs. Respondents believe this would 
build upon the observed improvement in Aboriginal/non- Aboriginal relations flowing from 
job placement involvement in some sites. 

• It was suggested that a better orientation structure for local coordinators and teachers . 
working with cooperative education was needed. Orientation activities should be similar 
to those prepared for coordinators who implement the program. 

• First Nations respondents suggested a dedicated coordinator to recruit employers, assist 
students, and communicate with parents and the school is essential for the Cooperative 
Education Program. For larger communities, a full-time coordinator should be hired while 
smaller communities might pool their resources and use one person to administer the 
Summer Student Career Placement Program and the Cooperative Education Program. 



Communication 

Respondents suggested that First Nations models of cooperative education and science programs 
be discussed at a national meeting to facilitate sharing of ideas on alternatives for administering 
the programs. One suggestion was to have a conference with First Nations cooperative education 
coordinators from across Canada. 

• Two communities volunteered to give presentations to other communities about their own 
programs in an effort to help them set up a new cooperative education program. 
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Resources 



Some First Nations respondents would like to see more resources and time allocated to 
cooperative education program orientation for teachers in schools. 

Respondents suggested that the government provide additional funds to cover the extra 
cost of insurance for students in job placements off-reserve. 

First Nations organizations and coordinators at the local level want DIAND to increase 
resources, so that for the Summer Student Career Placement Program youth can be 
transported to jobs outside of the community. 



Reporting and Evaluation 

First Nations respondents noted the need for a formal long-term evaluation. They expressed an 
interest in wanting to be involved in evaluating their own programs and directing such a study. 
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Section 8 - Conclusions and Recommendations 



This section presents conclusions and recommendations associated with the evaluation. Overall, 
the evaluation found that First Nations had positive experiences with the strategy and there is a 
strong view that short-term impacts have been achieved. In addition, many suggestions for 
improvements, such as establishing programs for out-of-school and unemployed youth, have been 
addressed by the addition of two new programs to the renewed strategy. Some First Nation 
officials, particularly representatives of the Implementing Organizations noted the need for 
increased resources to be devoted to the administration of the programs. In most cases, the 
adoption of the delivery of the strategy by these organizations is being done by utilizing existing 
human resources. 

While the strategy is viewed by all stakeholders as a success, there are still some areas that need 
improvement and will assist in making the strategy more effective. 



Implementation of the Strategy 

Most First Nations interviewed indicated that they were aware of and participated in the Summer 
Student Career Placement and Science and Technology Summer Camp programs. However, 
interviews with eligible First Nations schools indicate that communities require more information 
about the Cooperative Education Program. Those interviewed tend to look toward Aboriginal 
forums and communication material to assist them in developing a program. 

Recommendation 1 . 

It is recommended that increased activities and resources be devoted to capacity 
building among First Nations and Inuit and related organizations to support them 
in implementing the strategy. 

First Nations and D1AND both have a role in capacity building Areas idenfitied by the evaluation 
include: 



• information sharing, networking and communications 

• First Nations respondents offered many suggestions revolving around the sharing of 
information and building capacity among local coordinators responsible for delivering 
strategy programs. Best practices include monthly conference calls of program 
coordinators, newsletters, and regional and national workshops. In terms of 
communication materials, a best practice includes short information booklets prepared by 
a First Nations organization to explain the strategy to its member communities. 

• strengthening administrative and human resources. 
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Departmental and First Nations representatives noted resource constraints in administering the 
strategy at the local level. An increased focus on ensuring adequate resources are available for 
administrative and capacity building activities would strengthen the long-term implementation of 
the strategy. Best practices included having financial resources and experienced coordinators to 
administer the strategy’s program. Case studies show that coordinators, especially those 
responsible for the Cooperative Education Program, require strong administrative and program 
development skills to undertake activities such as recruiting employers and developing 
curriculum. Other best practices of sharing resources and expertise included smaller First Nations 
using one coordinator to administer all their youth programs, and hiring students from one 
program to help administer another strategy program. These activities should be encouraged 
across all regions. 



Delivery Models 

The evaluation examined the variety of delivery models currently being used across the country. 
Most respondents believe that a decentralized approach where First Nations organizations 
administer the strategy’s programs on behalf of communities appears to be the most effective. 
Case studies indicate that First Nations organizations are able to respond more quickly to set up 
programs because they use existing structures to administer them. First Nation organizations also 
appreciate the opportunity to enhance capacity building efforts which were found to occur less in 
those regions where DIAND, administers the strategy. 



Recommendation 2 : 

It is recommended that all DIAND regional offices continue their efforts to 
encourage First Nations and Inuit regional organizations to adopt the management 
of the strategy. To assist this, the department may need to consider providing a 
financial support to First Nation and Inuit organizations to administer the strategy, 
as has been done in the Quebec Region. 

The transfer of responsibilities reflects the general direction the department has taken with regards 
to local programming and reflects the staff complement in regional offices. While this approach 
will alleviate the administrative burden felt by DIAND regional coordinators, it may transfer this 
concern to First Nations organizations which in some cases are already reluctant to take on the 
administration of the strategy. The regions will need to engage in dialogue with First Nations to 
identify barriers or concerns associated with the management of programs and the associated 
administrative resources required. The department may need to consider providing a financial 
support to First Nation organizations to administer the strategy, as has been done in the Quebec 
Region. 
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The transfer will provide First Nations organizations with the opportunity to control the quality 
and direction of programs. The approach may also help create economies of scale by allowing 
First Nations to share resources, and by possibly lowering the overall amount First Nations in the 
region spend on managing the strategy, as has been done in Saskatchewan and Quebec. This 
approach may also allow for more exchange and sharing of ideas among First Nations 
communities. 

DLAND regions will need to engage in dialogue with First Nations to identify the most 
appropriate model for them and to resolve any concerns about administering the strategy. 
DIAND and First Nations may also wish to consider other models being utilized by other 
departments. For example. Human Resources Development Canada, Industry Canada, and 
Foreign Affairs and International Trade have agreements with the Assembly of First Nations for 
the delivery of components of their youth programming. Consideration should be given to 
consulting these third parties to obtain lessons learned and best practices from these models. 



Ongoing Reporting 

While DIAND respondents are confident that program funding has been spent appropriately, 
current reporting does not provide information to reflect this situation. The Statistical Overview 
found that reporting among regions is not consistent. The amount expended and the number of 
placements created are under-reported and substantially understate program outputs and 
outcomes in some regions. The annual First Nation financial audit can be used to verify the 
amounts expended on the programs. A best practice to encourage timely and consistent program 
reporting is where regions and/or First Nations regional implementing organizations have held 
back a percentage of funds 

The evaluation forms which collect information on the ongoing program outputs and outcomes 
were generally found to be sufficient; however, the evaluation identified a number of 
enhancements to improve the quality of reporting. These include the addition of information on 
the type of employers recruited to assess employment sectors where the programs have been 
most effective, and the type of skills/positions created. 

Recommendation 3 : 

DIAND regional offices should ensure that annual First Nations and Inuit audit 
reports are tracking Youth Strategy expenditures. It is recommended that the 
timeliness and level of submissions of program evaluation forms be improved. 
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Recommendation 4 : 



The evaluation forms should be enhanced to include information on the type and 
number of employers participating in the strategy, the types of skills/positions 
undertaken and the sources of additional funds contributed to the programs. 

Recommendation 5 : 

The department should work with First Nations and lnuit and other departments 
involved in Aboriginal youth programming to promote consistency in the reporting 
requirements across federal departments. 



Long-Term Performance Measurement 

First Nations and D1AND utilize performance information to assist in the management and 
delivery of the program to ensure that the programs in the strategy are effective and that the needs 
of youths are being met. First Nations noted a strong interest in undertaking evaluations of their 
programs and tracking information to assess the long-term impacts of the strategy, but noted that 
they are unsure of the type of informaiton to collect. On its part, DIAND needs to collect 
information to account to Parliament on the outcomes of the funds it manages. This need will 
grow as funding for the strategy has doubled. At the same time, regions and First Nations are 
reluctant to engage in extensive data collection and want to ensure that the long-term tracking of 
the program is in keeping with the size of the initiative. Therefore, DIAND needs to work with 
First Nations to decide which indicators it wants to measure and what is the most cost-effective 
approach. The evaluation framework accompanying this report identifies potential long-term 
indicators, associated information requirements, and approaches for its collection. 

A shared understanding of reporting is important as both DIAND and First Nations will use this 
information in decision-making. First Nations currently collect information on an informal basis 
but would benefit by consolidating efforts and formalizing the reporting process. This will allow 
First Nations the opportunity to share information with communities across Canada and to 
identify opportunities for program efficiencies and economies of scale. 



Recommendation 6 : 

It is recommended that DIAND identify, in consultation with First Nations and 
lnuit, performance measurement indicators that will assist both partners to measure 
the long- term success of the strategy. 
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Recommendation 7 : 



For the long-term evaluation of the strategy, it is reccommended that a partnership 
approach be undertaken by the department and First Nations and Inuit. This 
process should include Aboriginal youth and use the evaluation framework 
developed in the course of this current interim evaluation as a starting point. 

It is also suggested that DIAND continue to work with First Nations and implement any future 
evaluation in full partnership with them. 
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Summary of Regional Allocations for the 

Youth Strategy 1996-1997 



Summary of Regional Allocations for the 
Youth Strategy 1996-1997 


Region 


Summer Student Career 
Placement Program 


Science and 
Technology Camps 
Program 


First Nations Schools 
Cooperative 
Education Program 


Atlantic 


$359,900 


$48,000 


$96,000 


Quebec 


$646,640 


$96,000 


$544,000 


Ontario 


$1,294,460 


$136,000 


$512,000 


Manitoba 


$829,540 


$120,000 


$768,000 


Saskatchewan 


$710,360 


$104,000 


$608,000 


Alberta 


$702,100 


$112,000 


$416,000 


British Columbia 


$973,500 


$96,000 


$256,000 


Yukon 


$53,100 


$8,000 


• 


Northwest Territories 


$330,400 


$80,000 




Headquarters* 


$100,000 


- 


- 


Canada 


$6,000,000 


$800,000 


$3,200,000 


Source: DIAND headquarters data. February 1997. 

(*) Allocated to national First Nations organizations. 
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Action Plan 



INTERIM EVALUATION OF FIRST NA TIONS ANI) INUIT YOUTH EMPLOYMENT STRATEGY 

Preamble* The development of this action plan, its implementation and all activities related to First Nations and Inuit youth employment programming 
will be achieved through a partnership between DIAND, AFN and 1TC. A national working committee will be established comprised of 

three representatives from each partner. 




Terms of 
Reference 



96/26 



Terms of Reference 

Interim Evaluation of the DIAND Youth Strategy Program 



Background: The Department of Indian Affairs and Northern Development (DIAND) 

initiated the Youth Strategy Program in April 1996. The overall objective 
of the Strategy is to provide First Nations and Inuit youth with career 
related opportunities to assist entry into the labour force. 

The strategy consists of three distinct components: (a) the First Nations 
and Inuit Summer Student Career Placement Program; (b) the First Nations 
and Inuit Science and Technology Summer Camp Program; and, (c) the 
First Nations Schools Co-operative Education Program. The first two 
components began in the summer 1996 and the Co-operative Education 
Program commenced in September 1996. 

The Summer Student Career Placement Program provides wage 
contributions to assist students in obtaining career related work experience. 
The Science and Technology Summer Camp Program aims to promote 
science and technology as career choices. The Co-operative Education 
Program aims to establish or enhance existing co-operative education 
programs in First Nations schools. 

The initiation of the Youth Strategy Program became possible with the 
announcement in the March 1996 Budget, reallocating $315 million of 
budget savings ($105 million per year) to help create employment 
opportunities for young Canadians for the following three years. For the 
. 1996-1997 fiscal year, $10 million has been allocated to the DIAND Youth 
Strategy initiative. 

NEED: The department has committed to Treasury Board to undertake an interim 

evaluation of the initial implementation of the strategy. 

SCOPE: The interim evaluation will assess the initial performance and short-term 

impacts of the strategy and its components. 

Issues: The interim evaluation will address the following issues: 

• What is the status of program implementation and what are the 
critical phases of the administrative delivery systems? 

• How effective and efficient has the initial implementation of the 
strategy been? 
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• What have been the intended and unintended short-term impacts of 
the strategy? 

• What are the best practices and lessons learned? - 

• What improvements can be made and what are the options for 
future programming? 

Approach: The interim evaluation will be managed by a Senior Evaluation Manager 

from the Departmental Audit and Evaluation Branch (DAEB) in 
consultation with an evaluation advisory committee. This committee will 
consist of representatives of DIAND headquarters and two regional offices, 
the Assembly of First Nations (AFN) and a participating First Nation. 

The interim evaluation will be undertaken by DAEB through the use of in- 
house and contracted resources. The primary evaluation methods will 
consist of interviews with a sample of participating students. First Nation 
co-ordinators, and employers. Reviews will be undertaken at two to three 
DIAND regional offices and at headquarters. On the basis of the planning 
documents and findings of the interim evaluation, a performance 
measurement framework will be produced to assess long-term impacts for 
future evaluation. 

Resources & 

TIMEFRAME: It is estimated that the evaluation will cost up to $60K. An initial draft of 

the evaluation is to be completed by March 1996. 
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